78

7

The Transmission of Information

runner has to cover, the nature of the terrain, his physique, and so on. ³ Similarly, the

capacity of a heliograph signalling system (in flashes per minute) depends on the

dexterity of the operators working the mirrors and the availability of sunlight.

It is obviously convenient, when confronted with the practicalities of comparing

the capacities of different channels (for example, a general in the field may have to

decide whether to rely on runners or set up a heliograph) to have a common scale with

which the capacities of different channels may be compared. A channel is essentially

transmitting variety. A runner can clearly convey a great deal of variety, since he

could bear a large number of different messages. If he can comfortably carry a sheet

on which a thousand characters are written, and assuming that the characters are

selected from the English alphabet plus space, then the variety of a single message

is 1000 log Subscript 2 Baseline 27 equals 47541000 log2 27 = 4754 bits to a first approximation. If the runner can convey three

scrolls a day, the rate of transmission of variety is then3 times 4754 divided by left parenthesis 12 times 3600 right parenthesis equals 0.333 × 4754/(12 × 3600) = 0.33

bits per second, assuming 12 h of good daylight.

The heliograph operator, on the other hand, may be able to send one signal per

second, with a linear variety of two (flash or no flash); that is, during the 12 h of good

daylight, he can transmit with a rate of log Subscript 2 Baseline 2 equals 1log2 2 = 1 bit/s.

It may be, of course, that the messages the general needs to send are highly

stereotyped. Perhaps there are just 100 different messages that might need to be

sent. ⁴ Hence, they could be listed and referred to by their number in the list. Since

the number 100 (in base 10) can be encoded bylog Subscript 2 Baseline 100 equals 6.64log2 100 = 6.64 bits, any of the 100

messages could be sent within 7 s. Furthermore, if experience showed that only 10

of the messages were sent rather frequently (say with probability 0.05 each), and the

remaining 90 with probability StartFraction 0.5 Over 90 EndFraction ^0.5

90 ^{, the application of Eq. (6.5) shows that 5.92 bits}

would suffice, so that a more compact coding of the 100 messages could in principle

be found. ⁵

We note in passing, with reference to Eq. (6.13), that all of the details of the

physical construction of the heliograph, or whatever system is used, and including

the table of 100 messages assigning a number to each one, so only the number needs

to be sent, are included in upper KK. Should it be necessary to quantify upper KK, it can be done

via the algorithmic complexity (AIC; see Sect. 11.5), but as far as the transmission

of messages is concerned, this is not necessary, since we are only concerned with

the gain of information by the recipient (cf. Eq. 6.14).

The meaning of each message (i.e., an encoded number) sent under the second

scheme could potentially be very great. It might refer to a book full of instructions.

Here we shall not consider the effect of the message (cf. Sect. 6.3.3).

Another point to consider is possible interference with the message. The runner

would be a target for the enemy; hence, it may be advisable to send, say, three runners

in parallel with copies of the same message. It might also have been found that the

3 Note that here the information source is the brain of the originator of the message, and the encoder

is the brain-hand-pen system that results in the message being written down on the scroll.

4 Such stereotypy is extensively made use of in texting with a cell phone.

5 Note that Shannon’s theory does not give any clues as to how the most compact coding can be

found.